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EXAMINER'S ANSWER 



This is in response to the appeal brief filed 20 September 2004. 
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(1) Real Party in Interest 

A statement identifying the real party in interest is contained in the brief. 

(2) Related Appeals and Interferences 

A statement identifying the related appeals and interferences which will directly affect or 
be directly affected by or have a bearing on the decision in the pending appeal is contained in the 
brief. 

(3) Status of Claims 

The statement of the status of the claims contained in the brief is correct. 

(4) Status of Amendments After Final 

No amendment after final has been filed. 

(5) Summary of Invention 

The summary of invention contained in the brief is correct. 

(6) Issues 

The appellant's statement of the issues in the brief is missing. 

(7) Grouping of Claims 

The rejection of claims 1 and 3-5 stand or fall together because appellant's brief does not 
include a statement that this grouping of claims does not stand or fall together and reasons in 
support thereof. See 37 CFR 1.192(c)(7). 

( 8) Claims Appealed 

The copy of the appealed claims contained in the Appendix to the brief is correct. 

(9) Prior Art of Record 

5,577,131 ODDOU 11-1996 
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Shafarenko, L. et al. "Automatic Watershed Segmentation of Randomly Textured Color Images" 
IEEE Transactions on Image Processing, vol6, no. 11 (Nov. 1997), pp. 1530-1544. 

Wang, D. "Unsupervised Video Segmentation Based on Watersheds and Temporal Tracking" 
IEEE Transactions on Circuits and Systems for Video Technology, vol8, no. 5 (Sept 
1998), pp. 539-546. 

Luo, J. et al. "Incorporation of Derivative Priors in Adaptive Bayesian Color Image 

Segmentation" Proceedings of the International Conference on Image Processing, vol3 

(Oct 1998), pp. 780-784. 
Corridoni, J.M. et al. "Pyramidal Retrieval by Color Perceptive Regions" Proceedings of IEEE 

International Workshop on Content-Based Access of Image and Video Database (Jan 

1998), pp. 2-11. 
(10) Groun ds of Rejection 

The following ground(s) of rejection are applicable to the appealed claims: 

Claims 1 and 3-5 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Shafarenko in view of "Unsupervised Video Segmentation Based on Watersheds and Temporal 
Tracking" by Wang and U.S. Patent 5,577,131 by Oddou. 

Regarding claim 1, Shafarenko discloses a method of extracting regions of homogeneous 
color in a digital picture comprising: 

dividing the digital picture into blocks (Shafarenko operates on pixels, which are the 
smallest image blocks); and 
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merging together spatially adjacent blocks that have similar color properties to extract the 
regions of homogeneous color (Abstract: watershed routine is used to merge pixels according to 
color contrast). 

Shafarenko discloses the merging step comprises: 

extracting a feature vector for each block (Shafarenko processes pixels in LUV color 
space; the L, U, and V values for each pixel comprise a feature vector); 

estimate a scalar gradient value for each block as a function of the feature vector, the set 
of gradient values defining a color gradient field (Section B, paragraph 2, page 1533: each pixel 
is assigned an LUV gradient value according to the maximum Euclidean distance to the furthest 
neighbor; the set of all gradient values produces a field); 

segmenting the gradient field with a watershed algorithm that divides the gradient field 
into a set of spatially connected regions of homogeneous color (third paragraph, page 1531: 
watershed algorithm uses LUV gradient to segment image by color). 

Shafarenko is silent to digitizing the color gradient field. However, Shafarenko' s method 
is implemented on a computer, so any computed values are digital. 

Shafarenko is silent to preprocessing the digitized color gradient field to produce a 
smoothed color gradient field. 

Wang discloses a similar segmentation routine, wherein image gradients are applied to a 
watershed algorithm to segment an image into homogeneous regions. Wang teaches smoothing 
the gradient field prior to utilizing it for the watershed algorithm. Wang dilates then erodes the 
gradient image, thereby reducing local minima caused by noise or quantization error (Section A, 
step 3, page 540). / 
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It would have been obvious to one of ordinary skill in the art at the time of the invention 
to modify Shafarenko by Wang in order to preprocess the color gradient field as claimed, since 
Wang teaches smoothing the gradient field removes noise. 

Shafarenko and Wang are silent to dividing the digital picture into blocks, wherein each 
block comprises a plurality of pixels , and performing the claimed steps on the blocks, rather than 
individual pixels. 

Oddou discloses a system for segmenting images into homogeneous regions by applying 
a watershed algorithm to a gradient field in much the same manner as Shafarenko and Wang. In 
particular, Oddou discloses performing the segmentation routine on blocks of pixels rather than 
individual pixels. Column 8, lines 39-54: an image to be segmented is divided into macroblocks 
that are e.g. 16x16 in size; then parameters are extracted from each block based on the average 
value of pixels in each block, and the segmentation routine is performed on the basis of the 
parameters of each block. Thus, by averaging each 16x16 block of pixels and utilizing the 
average values for the segmentation, Oddou essentially reduces the resolution of the image by a 
factor of 16 and treats the 16x16 blocks as individual pixels. 

Although Oddou seeks to segment by texture rather than by color, the major advantage of 
operating on blocks of pixels rather than individual pixels to segment an image is apparent to 
those skilled in the art - processing time is reduced. 

It would have been obvious to one of ordinary skill in the art at the time of the invention 
to modify Shafarenko and Wang by Oddou to perform the steps for extracting regions of 
homogeneous color on blocks of pixels rather than individual pixels since operating on blocks of 
pixels rather than individual pixels is less computationally intensive. 



Application/Control Number: 09/496,068 
Art Unit: 2623 



Page 6 



Regarding claim 3, Shafarenko discloses the extracting step comprises: 

transforming the data in each block into a perceptually uniform color system (Section B, 
page 1533: pixels are placed in perceptually uniform LUV color space); 

calculate N moments of the data in each block for each color component, the set of 
moments being the feature vector for the block (the moment is simply the L, U, and V average 
values of the block, per Oddou's averaging of the blocks). 

Regarding claim 4, Shafarenko discloses the estimating step comprises: 

selecting the maximum of the distances between the feature vector of each block and the 
neighboring vectors as the gradient value for the block (Section B, paragraph 2, page 1533: each 
pixel (i.e. block) is mapped onto the distance to its furthest neighbor). 

Shafarenko does not expressly disclose obtaining distances between the feature vector of 
each block and the feature vectors of each neighboring block. However, in order to find the 
maximum distance, all of the distances must be known. Therefore, this step of obtaining is 
implicit in Shafarenko' s teaching. 

Regarding claim 5, Shafarenko teaches applying a weighted Euclidean distance metric to 
the feature vectors to determine the distances (Section B, paragraph 2, page 1533: Euclidean 
distance is used to estimate the gradient; weighting is unity since there is only one moment for 
each block). 
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Claim 8 is rejected under 35 U.S.C. 103(a) as being unpatentable over "Incorporation of 
Derivative Priors in Adaptive Bayesian Color Image Segmentation" by Luo et al. ("Luo") in 
view of "Pyramidal Retrieval by Color Perceptive Regions" by Corridoni et ah ("Corridoni"). 

Regarding claim 8, Luo discloses a method for representing spatial relationships between 
regions of homogeneous color in a digital picture (e.g. figure 1 : spatial relationship between 
homogeneous regions) producing data comprising the steps of: 

dividing the digital picture into blocks (i.e. each pixel constitutes a block); 

estimating a scalar gradient value for each block by defining a color gradient field 
corresponding to each block (see corresponding explanation for claim 7); 

representing said data corresponding to the digital picture as a probability distribution 
function calculated in view of blocks of the digital picture that are homogeneous in color and 
distances between the blocks that are homogeneous in color (Luo represents the image data as a 
maximum a posteriori (MAP) probability function using a Gibbs distribution in order to achieve 
a segmented image, as explained above for claim 7. Luo discloses that the MAP function is 
calculated in view of blocks of the digital picture (i.e. it is calculated based on the pixels of the 
digital picture). Luo also discloses that the MAP function is calculated in view of distances 
between the blocks (i.e. it is based on the first-order derivatives, which are derived from the 
gradient magnitudes, or distances between blocks (see section 2.5). It should be noted that pixels 
are homogeneous in color - they exhibit the same color throughout the area of the pixel. 
Therefore, every block of Luo is homogeneous in color). 

Luo is silent to the data being suitable for use in an image database application. 
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Corridoni discloses a region-based image querying system, whereby images are 
segmented into regions of homogeneous color at different levels of resolution, and those regions 
are used for matching queries and retrieving images in an image database application. In 
particular, Corridoni discloses segmenting an image into regions of coherent color (section 2), 
describing the features of each of the regions (section 2.1), and allowing a user to form queries 
for searching for database images based on the region descriptors (section 3). Thus, Corridoni 
teaches the color segmentation of an image provides data that is suitable for use in an image 
database application. 

Therefore, Luo's data, which represents the homogeneous color regions of an image, are 
suitable for use in an image database application, since Corridoni teaches that a color 
segmentation of an image produces data that is suitable for searching for the image in a database 
application. 
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(11) Response to Argument 

Regarding claim 1, Appellant argues that Shafarenko teaches away from processing 
images on the basis of blocks of pixels. Appellant relies on a statement in Shafarenko 's 
introductory section as providing evidence that Shafarenko' s segmentation processing would be 
inoperable if it is performed on blocks rather than individual pixels: "if one tries to filter out 
significant features by applying size-based filters such as opening or closing [2], or their 
combinations, then small but significant features in terms of color saliency may be removed" 
( emphasis added ). 

The above quote recognizes that there is a trade-off associated with filtering an image to 
be segmented. On the one hand, it is desirable to filter out insignificant features, such as noise or 
the like, which are unimportant and should not be subject to segmentation. However, in the 
process of filtering these insignificant features, one may unintentionally filter out fine details that 
are intended to be a part of the segmented image. 

Appellant asserts that one skilled in the art would recognize this passage as teaching 
away from modifying Shafarenko to perform her method of segmentation via block processing. 
Examiner maintains that this assertion is false. The quote merely recognizes there are pros and 
cons associated with filtering an image before segmentation and does not preclude one skilled in 
the art from modifying Shafarenko as suggested by the Examiner. 

Shafarenko is concerned with segmenting randomly textured images, such as granite. 
Therefore, it may be incumbent for Shafarenko not to process the image in blocks, or to process 
the image only in small blocks. However, there is no teaching or suggestion in Shafarenko that 
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her method of segmentation would be inoperable if it were processed on blocks of pixels rather 
than individual pixels. Oddou suggests performing watershed segmentation on textured images 
in blocks that are of size "p x q" (column 8, lines 39-54), which implies that the exact size of the 
blocks are arbitrary and dependant on the specific application, performance criteria, or type of 
image to be processed. 

Appellant also argues that the modification of Shafarenko by Oddou is invalid because 
Oddou segments by texture rather than by color. However, the only teaching relied upon in 
Oddou is that, when performing watershed segmentation on a textured image, the segmentation 
process may be advantageously effected on the basis of blocks of pixels rather than individual 
pixels. Oddou generates blocks of pixels and then extracts a feature from the block (such as the 
mean value). Further processing is carried out on the basis of this feature value. Similarly, 
Appellant's claim calls for dividing an image into blocks and extracting features from the blocks; 
the features are used as a basis for further segmentation processing. 

The fact that Oddou segments by texture rather than color has little to do with the 
proposed modification to Shafarenko to process blocks of pixels rather than individual pixels. 
Both Shafarenko and Oddou perform watershed segmentation routines on textured images in 
order to divide the images into homogeneous regions. It is unclear how one skilled in the art 
would be precluded from utilizing Oddou' s teaching of segmenting a textured image on the basis 
of blocks simply because Oddou and Shafarenko do not seek to achieve exactly the same results. 

Regarding claim 8, Appellant argues that the modification of Luo by Corridoni is invalid 
because Corridoni does not disclose segmenting an image by color in exactly the same way as 
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Luo. However, Corridoni was only relied upon for the broad teaching that images segmented into 
homogenous color regions are "suitable for use in an image database application." See e.g. § 2 of 
Corridoni, wherein images are segmented into non-overlapping regions of color, so that the 
image data is more suitable for search and retrieval (i.e. "database") applications. 

Furthermore, since Luo teaches all of the claimed steps of representing spatial 
relationships between regions of homogeneous color, one skilled in the art would conclude that 
an image segmented in such a way is inherently suitable for use in a database application, based 
merely on the fact that there are no material differences between Luo's method and the claimed 
method, save for the "suitable for use" qualification. 

Nevertheless, even if one were to conclude that Luo's color-segmented images were 
somehow "unsuitable for use in a database application," this feature amounts to no more than an 
intended use of the claimed method. It is well-established that, when the prior art discloses all of 
the limitations of a claim except for its intended use, then the intended use is not given patentable 
weight, and the prior art anticipates the claim. See e.g. In re Shreiber. 44 USPQ2d 1429 (1997). 
Corridoni was relied upon to conclusively demonstrate that Luo's segmented image data is in 
fact "suitable for use in an image database application " but the application of Corridoni, in light 
of Shreiber , was not required. 



For the above reasons, it is believed that the rejections should be sustained. 
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